Sorting a REALLY BIG delimited text file in UNIX / VMS [closed]

Posted by gunbuster363 on Programmers See other posts from Programmers or by gunbuster363
Published on 2011-02-09T07:34:06Z Indexed on 2011/02/09 15:33 UTC
Read the original article Hit count: 281

Filed under:

sorting

Hi everyone, I am going to sort a REALLY BIG delimited text file, say 250Mb (or a bunch of files of more or less than 250Mb) . It have 37 fields, and I need to sort it by 5 fields, for example 1st, 4th, 5th, 6th 7th fields. Under Unix / VMS, do I have a good option to do this FAST? I can write COBOL program.
Now I am trying to sort them using the below command, but it already run for a long time and just not going to finished.

Thank you.

The command I used: time sort -t ',' -o sorted.txt +0 -1 +4 -5 +5 -6 +6 -7 +22 -23 *.DAT_gprscdr_ftpd

Related posts about sorting

Sorting eigenvectors by their eigenvalues (associated sorting)

as seen on Stack Overflow - Search for 'Stack Overflow'
I have an unsorted vector of eigenvalues and a related matrix of eigenvectors. I'd like to sort the columns of the matrix with respect to the sorted set of eigenvalues. (e.g., if eigenvalue[3] moves to eigenvalue[2], I want column 3 of the eigenvector matrix to move over to column 2.) I know I can… >>> More
Sorting in SSRS Reports overriding Group sorting

as seen on Stack Overflow - Search for 'Stack Overflow'
I am new to SSRS (2005) and am creating my first report. I need to create a group-based running value but is sorted or some other field. For example, from the data below |Employee| Day |Hours|Salary| |E1 | 1.1 | 5 | 5 | |E2 | 1.2 | 6 | 6 | |E3 | 1.3 | 7 | 7 | |E2 … >>> More
org-sort multi: date/time (?d ?t) | priority (?p) | title (?a)

as seen on Super User - Search for 'Super User'
Is anyone aware of an org-sort function / modification that can refile / organize a group of TODO so that it sorts them by three (3) criteria: first sort by due date, second sort by priority, and third sort by by title of the task? EDIT: I believe that org-sort by deadline (?d) has a bug that… >>> More
A Look at the GridView's New Sorting Styles in ASP.NET 4.0

as seen on Internet.com - Search for 'Internet.com'
Scott Mitchell looks at four new sorting-related style properties added to the GridView in ASP.NET 4.0. >>> More
A Look at the GridView's New Sorting Styles in ASP.NET 4.0

as seen on 4 Guys From Rolla - Search for '4 Guys From Rolla'
Like every Web control in the ASP.NET toolbox, the GridView includes a variety of style-related properties, including CssClass, Font, ForeColor, BackColor, Width, Height, and so on. The GridView also includes style properties that apply to certain classes of rows in the grid, such as RowStyle,… >>> More

Developer IT